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ABSTRACT 


In 1979, a F-S aircrew training program, designed using the 
Instructional Systems Development process, was implemented in the two P-3 
Fleet Replacement Squadrons. One squadron adopted workbooks 
recommended by ISD, one continued with lectures. This thesis compares 
the cost differences arising from *wo different methods of instruction now 
in use in P-3 training progcams. A conceptual model is developed to 
compare the opportunity costs associated with the selection of one training 
method over another. The results indicate that the lecture method of 


instruction is least costly. 
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I. IN“ xODUCTION 


This thesis will compare the cost differences arising from two 
different methods of instruction, a technology based method that uses 
workbooks as the primary means of information transfer, and a 
conventional instruction method that uses lectures. The purpose of the 
comparison is to determine which method has higher life-cycle costs. 
Because the selection of one training method over another has opportunity 
costs associated with it, many of those costs will also be discussed in an 
attempt to provide the decision maker with a complete description of 
pecuniary and non-pecuniary costs which are relevant to training method 
selection. 

Efficient and effective training is of critical concern to the Navy 
because of the thousands of men and women that must be trained 
annually. Navy training systems typically cost more than civilian 
programs because Navy students are paid while attending training. 
Additionally, the consequences of poor training are high in the Navy 
because of the ramifications of poor performance while serving in one of 
the many billet's with critical responsibilities. [Ref. 1: pp. 1,3] 

The high costs of military training programs drive the need for the 
military to weigh carefully alternative methods of instruction so the lowest 
cost alternative that accomplishes the desired level of training 
effectiveness can be selected. Kearsley, in his book Training and 


Technology, suggests that technology based training programs are almost 


always cheaper in the long run than a conventional method of instruction 
[Ref. 2: p. 153]. Though this perception may sometimes be true, an 
appropriate training method should be selected only after a cost- 
effectiveness study closely examines each method. Unfortunately, because 
of the unique requirements of different training programs, detailed cost 
data for ready comparison is lacking [Ref. 3: p. 22]. 


A. BACKGROUND 
l. Military Training and Readiness 
The military has training programs because its readiness is 
critical to its mission. Military readiness is dependent on personnel 
readiness which in turn is affected by training, job experience, leadership, 
motivation, selection and other factors. Training design technologies have 
been applied to military training programs in an effort to improve 
readiness. (Ref. 1: p. 1] The technology commonly used in the Navy for 
designing training programs is a method called Instructional Systems 
Development. 
2. Instructional Systems Development in the Navy 
Instructional Systems Development (ISD) was mandated for 
training program development by the Chief of Naval Operations in the mid 
1970's. The ISD method is a performance based system that uses a step 
by step procedure to design and develop training programs by first 
identifying the required on-the-job performance. Following performance 
determination, job tasks are determined from which training objectives can 


be developed. In later steps the training objectives are sequenced in a 


logical order, the delivery media are selected and developed, and the 
program is implemented. The final stage of the ISD process is a control 
phase based on continuing internal and external evaluation to revise the 
training. 
3. ISD and the P-3 Aircraft Training Programs 
The Navy contracted with a civilian firm in 1975 to revise the 


aircrew training program for the landbased P-3 anti-submarine warfare 
aircraft using ISD methods [Ref. 4: p. v]. Revision was felt to be essential 

because of numerous shortcomings is the training program that were | 
identified in an audit conducted by the P-3 training squadron and the 


Naval Training Device Center [Ref. 5: pp. 13,14]. The Chief of Naval 


Operations directed that the new training program follow a format similar 
to the one used for the S-3A aircraft, which is a carrier based aircraft | 
with a mission similar to the P-3 [Ref. 4: p. 1]. The new P-3 aircrew 
training program was intended to replace existing training at the two P-3 
Fleet Replacement Squadrons (FRS), Patrol Squadron Thirty-One (VP-31) 
located at Naval Air Station (NAS) Moffett Field, California, and Patrol 
Squadron Thirty (VP-30) located at NAS Jacksonville, Florida. 
4. Media Selection for the P-3 FRS 

During the ISD process, the instructional media for the training 
program were selected. The media selected were workbooks, slide-tapes, 
videotapes, and lectures/seminars. (Ref. 4: pp. 9,b1-b8] The workbooks 
were structured in a manner similar to programmed instruction, which 


allows students to proceed through the units at their own pace. However, 


due to the need for group instruction in simulators, aircraft labs, and 
flights, the workbook curriculum remained group-paced. 

As time went on, using a static medium (workbooks) to present 
frequently changing information led to a high number of content errors 
in many workbooks. The yroduction process used to update the 
workbooks could not make workbook changes as fast as they were 
occurring. Because of dissatisfaction in the workbook method, VP-30, the 
east coast training squadron, changed its training program to a lecture 
based system. VP-31, the program manager for the curriculum, continued 
to use the workbooks as the primary method of instruction but added 


lectures as necessary to make up for the workbook shortcomings. 


B. RESEARCH QUESTIONS 

This study compares the costs of the two methods of instruction used 
in the United States Navy's P-3 aircraft Fleet Replacement Squadron's. 
One method uses workbooks, which require a complex production system 
to maintain and update materials but has the potential for reducing 
manpower requirements. The other method is a conventional training 
system that uses lectures. Since the fleet squadrons of the east and west 
coast continue to operate at high levels of readiness even though the 
graduates that join the squadrons are taught with different instructional 
media, this study assumes that both training commands are meeting the 
minimum required level of effectiveness, and that the effectiveness of both 
training commands is the same. The costs used in this study include only 


those that are related to the classroom training methods. Other costs 








such as aircraft, fuel, spare parts, aircraft maintenance, simulators, and 
Squadron administration are not examined. 

This study gives a detailed description of the data required to 
measure the costs of a training method. Two different methods of 
instruction used in the Navy are compared to determine which method is 
more costly over the life-cycle of the course. The opportunity costs of 


various attributes associated with the training media are also compared. 


C. ORGANIZATION OF THE STUDY 

Chapter II presents a brief historical background of Instructional 
Systems Development and its subsequent use to develop the P-3 training 
program, and discusses several problems arising from the media chosen 
during the Instructional Systems Development process. Chapter III is a 
literature review of works that are pertinent to this study. In chapter 
IV, the cost model used to compare training alternatives is developed. 
Chapter V presents the data and results from using the cost model. A 


summary and conclusions are presented in chapter VI. 








Il. HISTORICAL DEVELOPMENT 


This chapter presents a brief historical account of the events leading 
to the training programs now in use at VP-3l1 and VP-30, and a 
description of the Instructional Systems Development process upon which 
the training programs were based. The historical development is useful 
because it illustrates how two programs, though based on the same 
training development methodology, can differ significantly in their teaching 
methods. It also serves as a reminder of some of the problems that can 
arise when adopting a new training methodology. Since the costs of the 
training methods of the two squadrons will be compared in this study, 
this discussion provides a useful framework for understanding how those 


cost differences developed. 


A. INSTRUCTIONAL SYSTEMS DEVELOPMENT IN THE MILITARY 

Because of the importance of training in the military, the military 
services have been active participants in the development and use of new 
training technologies. A curriculum development technique called 
Instructional Systems Development (ISD) evolved from the belief that the 
systems analysis approach, developed in the 1940's, could be used to 
simplify training program development. This technique has been the 
required standard for training program design in the Navy since the mid 


1970's. [Ref. 1: p. 2] 











The ISD approach gained popularity in all military service branches 
during the 1960s, and has been the most pervasive and influential of the 
training development methods [Ref. 2: p. 83]. Without a formal approach 
to training development, media selection and course development is more 
of an art. The ISD method turns course development into a sequenced 
procedure that identifies training requirements and uses set procedures 
to ensure that the training program meets the training objectives. Two 
major goals of ISD are: 

to make training (1) job relevant (meaning it would ultimately prepare 
the trainee for his or her function[s] in Navy readiness) and (2) 
cost-effective and efficient (meaning it would use the most efficient 
training methods to do the training). [Ref. 1: p. 4] 
Although ISD is often mistakenly confused with teaching methods or 
techniques it is much broader than that. Teaching methods are selected 
during one phase of the ISD process, and a training program developed 


with ISD could include several teaching methods such as lectures, self- 


paced study, or computer based instruction. [Ref. 1: p. 4] 


B. THE PROCESS OF INSTRUCTIONAL SYSTEMS DEVELOPMENT 
1. The Instructional Systems Development Method 
Kearsley, in his book Training and Technology, describes ISD as 
"a set of procedures for developing instructional programs in a consistent 
and reliable manner." (Ref. 2: p. 83] Developing training requirements 
based on explicit job performance criteria is the cornerstone of the ISD 
process. The content of a training program is determined only after an 


in-depth study of the job tasks of the position to be filled. [Ref. 2: p. 83] 


Differences between ISD and the traditional method of training program 


development are summarized in Table 2.1. 


TABLE 2.1. COMPARISON OF INSTRUCTIONAL SYSTEMS DEVELOPMENT 
AND TRADITiONAL APPROACHES TO INSTRUCTION 





ISD 
Approach 


Content is based upon 
performance requirements. 


Instructional objectives are 
explicitly stated. 


Content is developed by 
multidisciplinary team. 


Evaluation is part of development 
process. 


Media, strategies, and sequencing 
are analytically derived. 


Materials take into account 
students capabilities and 
differences. 


Development process is 
consistent and reliable across 
individuals. 





(Ref. 2: p. 84] 


Traditional 
Approach 


Content is based upon topics to 
be covered. 


Instructional objectives are 
implicit. 

Content is primarily developed 
by instructor. 


Evaluation occurs after training 
is implemented. 


Media, strategies, and sequencing 
are intuitively selected. 


Materials are designed for ideal 
or average student. 


Development process is 
idiosyncratic and varies with 
instructor. 





Training program development in the Navy using the ISD method 


was set forth from a series of documents published by the Navy as 


Interservice Procedures for Instructional System Development (NAVEDTRA 


106A). Figure 2.1 depicts the ISD process as described in NAVEDTRA 106A. 


In this model, five phases are used to develop a training program. The 


ANALYZE DESIGN 










tt 
ANALYZE 
308 


st. 1 
O€VELOP 
OBJECTIVES 






Y 


12 
SELECT 


TASKS! 
FUNCTIONS 





Vv 


i 3 
CONSTAUCT 








w 3 
OESCAIBE 
706 tnTAyY 
PEAFORMANCE BEHAVIOR 


MEASURES 











af Vv 


14 tla 
ANALYZE OE TERMINE 
EXISTING SEQUENCE & 


COURSES STAUCTURE 


Y 


+5 

SELECT 
INSTAUCTIONAL 
SETTING 


DEVELOP 











IMPLEMENT CONTROL 



























wi $ iv 1 vi 

SPECIFY IMPLEMENT CONOUCT 
LEARNING INSTRUCTIONAL INTERNAL 
EVENTS? MANAGEMENT EVALUATION 





ACTIVITIES PLAN 







Y Y 


conNouUCcT 
EXTEINAL 
EVALUATION 


Y 


SPECIFY 
INSTRUCTION 
MANAGEMENT 
PLAN & 
DELIVERY 
SYSTEM 


Vv 


REVIEW/SELECT 
EXISTING 
MATEATALS 


Iv 2 V2 
conouct 
INSTRUCTION 





Y 
V3 


REVISE 
Systtm 


v 


it 4 
O€VELOP 
INSTRUCTION 










Y 


st, $ 
VALIDATE 
INSTAUCTION 


Adapted from Ref. 5: p. 26. 
Figure 2.1. Five Phases of the ISD Model 


first and most important phase is the analysis phase. MThis phase 
determines if a need exists for the training in the first place, and, if the 
need exists, what will be taught. Job analysis, during this phase, 
identifies the criteria of effective job performance. These criteria becuine 
the basis for the material to be taught. [Ref. 6: p. 1.1] 

The course of instruction is designed during the second phase 
using the material from the analysis phase. The objectives of the course 
are identified, and the sequence of instruction over which the learning 
objectives will be presented is determined. Criterion referenced tests that 
measure the degree to which the students have attained the learning 
objectives are written. These tests are designed to determine the 
student's performance relative to the external job requirements. [Ref. 6: 
pp. 2.1,2.3] 

During phase three, the different learning strategies which will 
be used to teach the students the knowledge required by the performance 
based objectives are specified. The knowledge objectives are those that 
require the student to recall information but not apply it directly. 
Performance objectives require the student to use the information learned 
to solve a problem or operate equipment. 

Also in phase three, a curriculum outline is made by the program 
developers to aid in planning the course, and it subsequently serves as 
a curriculum approval document. Instructional materials are also 
developed. Writing the tests before developing the instructional material 
helps keep the program lean in the sense of limiting the instruction to the 


material needed to develop student proficiency in the objectives. 
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Validation of the instructional materials is conducted through 
internal review by the designers, individual student trials using rough 
drafts of the training materials, and, finally, group trials using rough 
drafts that have been changed to reflect any problems noted during 
internal and individual validation. Final validation is conducted on a 
regular class in the normal training environment, and measures student 
entry level behavior and course completion performance. [Ref. 6: pp. 
3.1,3.70,3.100,3.174-3.186] 

Phase four is the implementation phase. Here, the procedures 
used to conduct the course, such as scheduling, instructing, and 
monitoring, are specified. The final phase, phase five, is the controlling 
phase of the ISD program. In this phase, the training program is 
continuously evaluated with respect to meeting the training requirements 
using internal measures such as student and instructor feedback reports, 
and an in-depth annual course review. An external evaluation is 
conducted to determine if the graduates can do the job for which they 
were trained. External evaluation methods can range from informal 
discussions between the course instructors and those who supervise che 
course graiuates, to surveys and structured interviews. [Ref. 6: pp. 
4.1,5.1-5.32] 

2. ISD as Applied to the P-3 Training Squadrons 

In the mid 1970's, Courseware, Incorporated, was contracted to 

redesign, develop, implement, and evaluate the P-3 aircrew training 


program. Courseware worked with a Navy and Civil Service team of 


abt 





subject matter experts and training specialists to develop the new program 
using ISD methods. [Ref. 4: pp. 1,2] 

To select the instructional media for the P-3 training program, 
a media selection algorithm adopted from the S-3A training program was 
used. To use the algorithm, answers were obtained from a series of five 
questions for each of the objectives being taught. The questions 
identified the expected student behavior during testing (such as recall, 
recognition, or rule using), the level of content being taught (such as 
familiarization or memorization), the display requirements of the material, 
and the extent of the memorization requirements. The algorithm output 
provided a priority listing of the optimal media for delivery of the 
instructional materials. In many instances, computer based instruction was 
the first choice. However, because of cost constraints, computer based 
instruction was rejected and the instructional media selected were 
wor vooks, slide-tapes, videotapes, and seminars. [Ref. 4: pp. 9,b1-b8] 

The P-3 Instructional System Management Plan, which was the 
implementation document for the newly developed training program, shows 
that workbooks were the principal media for information transfer, and that 
lectures were not developed under the original ISD plan. Seminars, not 
to be confused with lectures in which an instructor teaches new material, 
were included in the training plan. Seminars were originally conceived as 
informal and impromptu get-togethers to provide an opportunity for the 
instructor to present updated information not yet included in the 
workbooks, and to encourage students to discuss areas from the 


workbooks in which they were having difficulty. [Ref. 7: pp. 5,6] 
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C. MEDIA DEVELOPMENT 

Workbook development for the P-3 training squadrons required a 
massive effort by the training program development team. Providing 
lessons in the form of workbooks for all P-3 aircrew positions meant 
creating roughly 850 different lessons averaging 35 pages per lesson [Ref. 
7: p. k-5]. Using workbooks. also meant that a major production system 
had to be established to support initial development and the later 
revisions resulting from changes to the aircraft and operating procedures. 
The production system included an educational specialist, editors, 
secretaries, a graphics shop, a print shop, and a library. 

Once the materials were developed, the validation phase should have 
been conducted. Because of delays in workbook production and the lack 
of suitable test subjects, only about five per cent of the instructional 
material was validated [Ref. 4: p. 15]. Even though procedural changes 
were made based on the validation of this small sample of materials, doubt 
was cast on the remaining materials. Had all of the material been validated 
perhaps some of the future problems encountered with the workbooks 
could have been avoided. 

With the same training program and media to be used for the east 
and west coast P-3 training squadrons, one squadron, (VP-31), was 
assigned model manager reSponsibilities for the training program. As 
model manager, VP-31 was responsible for curriculum oversight, which 
included producing and updating the training materials for the two 
training squadrons. The production system was set up in hangar one at 


NAS Moffett Field, California. Workbook production incorporated the 
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resources of two commands, VP-31 and Fleet Aviation Specialized 
Operational Trceining Group, Pacific Fleet. VP-31 formed an ISD department 
that was responsible for "ensure[ing that] courseware materials (including 
testing instruments) for all syllabi are developed and maintained using the 
most current content information available." [Ref. 7: p. 22] The Specialized 
Training Group's responsibility was to provide sevretarial and graphics 
support for ISD materials, to print ISD materials as required for both Fleet 
Replacement Squadron's, and to issue ISD materials to students attending 


the training at VP-31 [Ref. 7: p. 25]. 


D. EVOLUTION OF THE P-3 TRAINING PROGRAMS 

The training program developed using the ISD process was 
implemented at VP-31l in January of 1979, and at VP-30 in January of 1980 
[Ref. 5: p. 10]. By 1984, both training commands had added substantial 
lecture time because of problems with the workbooks. Numerous errors in 
the workbooks fostered frustration and a lack of confidence in the 
workbook system. These errors were the result of a production system 
originally set up to handie a much smaller workbook revision load. The 
inadequate staff was quickly overwhelmed by the changes needed because 
of aircraft modifications, changes in operating procedures, or changes in 
the tactical employment of the aircraft. The volume of changes led to 
lengthy backlogs in workbook revisions--some revisions required over a 
year to move through the system. In an instructional systems status 
letter, the Commanding Officer, VP-31 stated " ... by early 1983 the 


quality of the information contained in the system had slowly deteriorated 
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to a point that the courseware no longer reflected current operating 
procedures." [Ref. 8: pp. 1-4; The lectures were added to the training 
program to compensate for the workbook shortcomings because they could 
easily incorporate current information. An added bonus was later noted 
by a VP-31 commanding officer whe observed that lectures and workbooks, 
along with other media in the curriculum, lead to a richer mix of media 
and a greater opportunity for the students to grasp the objectives. 

Presently, VP-31 is predominantly workbook oriented, iLe., workbooks 
remain the primary media for information transfer. A large number of 
lectures have been added, however, so that the present ratio of workbooks 
to lectures is one to one, measured over all aircrew tracks. VP-30 has 
changed its curriculum so that it is a lecture oriented training program. 
Many of the workbooks are still used at VP-30, but only for supplemental 
reading assignments. 

As part of the evolutionary process within the training command, 
subject matter experts are changing the workbooks so that material 
subject to frequent change is deleted. With this material deleted, the 
student is referred to appropriate source publications where the material 
is more frequently updated. This new method should reduce the revision 
requirements, shorten the revision backlog, and reduce the student and 


staff frustration resulting from out-of-date materials. 
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II. LITERATURE REVIEW 


A. COST ANALYSIS METHODS 
1. Cost-Benefit Analysis 

Over the past 20 years there has been a dramatic increase in 
the analytic studies that are conducted to support public policy decisions. 
Some of the generic names applied to such analyses are environmental 
impact statements, cost-benefit analysis, risk-analysis, and decision 
analysis. These analyses, supposedly explicit evaluations of risks and 
benefits, theoretically provide the tools for “regulatory openness." Clear 
evaluations of policies required that the assumptions and numbers used to 
derive the estimates also be made available. Unfortunately, as emphasized 
by Firschhoff in The Art of Cost-Benefit Analysis, “these analytic 
techniques have both inherent limitations and potential for misuse." [Ref. 
9: p. 1.1] For analyses to be useful, the interested observers should be 
aware of the techniques that were used and know when the conclusions 
should be heeded. [Ref. 9: pp. 1.1,1.2] 

The basic premise of a cost-benefit analysis is that benefits and 
costs should be measured for the proposed action, and that a new 
technology or policy should be adopted only if the anticipated benefits 
outweigh the anticipated costs. The major challenge of a cost-benefit 
study lies in the correct and complete enumeration of all costs and 


benefits. [Ref. 9: p. 2.1] 
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2.  Cost-Effectiveness Analysis 
Because the quality of graduates from Navy training programs 
affects the Navy's readiness, training program effectiveness is a major 
concern. Most of the studies conducted to evaluate or compare the 
effectiveness of military training programs have used a cost-effectiveness 
approach. The studies are often undertaken to determine program cost or 
program effectiveness and not to determine if the benefits outweigh the 
costs. Often, the benefits of the military training program(s) being 
measured or compared are already assumed to outweigh the costs. 
Cost-effectiveness analysis provides a means of assessing decision 
alternatives among training programs. When more than one means of 
achieving an objective is available, cost-effectiveness analysis theoretically 
provides a means of identifying the least cost alternative. Orlansky 
suggests addressing the following three questions in a cost-effectiveness 
study (Ref. 3: p. 6]: 

1. Does the new technology, device, or method of instruction actually 
improve the effectiveness of training and, if so, by how much? 

2. How much does the new technology, device, or method of 
instruction cost (preferably on a life-cycle basis) compared to 
present practice? 

3, If the effectiveness of training is actually increased, is the amount 
of improvement worth its cost? 

These questions suggest several possible combinations of cost and 
effectiveness. The possible combinations, along with the decision each 


combination supports, is shown in Figure 3.1. If a training alternative is 


judged to have greater effectiveness at less cost or the same effectiveness 














at less cost it would appear to be a preferable choice. If the alternative 
has less effectiveness at the same or greater cost, or the same 
effectiveness at higher cost, rejecting the alternative would be the 
preferred choice. The choice is less clear cut when cost and effectiveness 
are both determined to be less, the same, or more than the other 
alternatives being compared. If cost and effectiveness are both higher, 
it must be determined whether the higher effectiveness justifies the 
higher cost. If the effectiveness is lower but cost is also lower it then 
becomes a question of whether the lower effectiveness is acceptable. If 
no difference in cost and effectiveness exists among alternatives, then 
final selection will depend on other important features such as the 
flexibility of the method under different student loading or instructor 


manning scenarios. [Ref. 3: p. 6] 


EFFECTIVENESS 





Adapted from Ref. 3: p. 7. 
Figure 3.1. Decision Matrix for Cost and Effectiveness 


The questions about costs and effectiveness presented by 
Orlansky imply that other features such as flexibility to operate under 


different student loading, do not have a cost. However, close observation 
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of a training program may reveal ways to measure or estimate the costs 
of “other features" that are often not included in a cost-effectiveness 
evaluation. Many features that are often overlooked by the analyst are 
features of the training program that have an opportunity cost associated 
with them. 
3. Opportunity Cost of an Alternative 

When making a choice between alternatives, there is an 
opportunity cost involved in that choice. To have something, other things 
must have been given up, or an opportunity to acquire an alternative 
thing must have been forgone. The forgone alternative is called the 
opportunity cost of that choice. Resources used on one project, for 
example, are no longer available for other uses. When operating with 
limited resources, a decision to proceed with one project has an implied 
decision not te undertake other projects. [Ref. 10: p. 110] This means 
that if all resources are used on the operational front, nothing remains for 
training, research, development, and logistics. At a less aggregate level, 
the choice to buy one more aircraft may mean fewer flight hours will be 
allocated to operate the aircraft. It is therefore in the opportunity cost 
where the real cost of the purchase is found. Using opportunity costs 
rather than strict accounting costs can help the decision makers determine 


the true costs of the alternatives during the decision process. 


B. COST-EFFECTIVENESS STUDIES COMPARING MILITARY TRAINING MEDIA 
The common definition of effectiveness, as used in military cost- 


effectiveness studies, is described by Hall, Rankin and Aagard as"... 
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(measures of) the degree to which a training course or system achieves 
the goals established for it." To measure effectiveness, the training goals 
must be converted into behavioral terms that can be measured, e.%. 
performance. [Ref. 11: p. 12] 

Many studies that have compared the effectiveness of different 
instructional media used in military training commands have concluded that 
no medium ranks significantly higher than the others in effectiveness. 
This result could be because most of the military's training consists of 
mixed media programs. Although one medium, such as lectures or 
workbooks, may be used more than the others, additional media or 
instructor attention may compensate for its shortcomings. 

A comparison between individual and conventional instruction was 
made by Freda and Hall [Ref. 13] in an attempt to determine the 
differences in effectiveness and efficiency. Data for the study were 
obtained from 19 military training programs and over 5000 students. Their 
methods for the comparison involved statistical regression using data 
obtained on five predictor and four criterion variables. The five predictor 
variables were: the method of instruction, (conventional or individualized), 
the trainees’ ability level, (based on armed forces qualification test 
scores), the type of training task, (fact, category, procedure, rule, or 
principle), gender of the graduate, and the location of the school. The 
four criterion variables chosen were divided into effectiveness measures 
and efficiency measures. The effectiveness measures were end-of-course 


grades and training adequacy ratings (performance survey from graduate's 
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first level supervisor). The efficiency measures were cost of the course 
and time to complete the course. 

Freda and Hall concluded that individualized and conventional 
instruction were equally effective ways of preparing sailors for their jobs. 
Within the two categories, students of individualized instruction with 
greater ability (mental categories 1 and 2) had higher course grades and 
finished in a shorter time than lower mental category students, but in 
conventional instruction, no differences were seen among mental categories. 
[Ref. 12: p. 69] 

Freda and Hall were not able to make a definitive judgement that one 
training method was more expensive than the other. For the co:.-es 
compared, the cost of individual instruction was higher than that of 
conventional instruction, but the courses compared were of different 
lengths and were not similar in type of material. To determine costs, the 
researchers used the "per capita course costing data base" from the Chief 
of Naval Education and Training. This data base does not contain 
research and development costs and also lacks many of the investment 
costs, so the comparison is primarily based on differences in student and 
instructor salaries. [Ref. 12: pp. 69,D.1] 

Braby and Evans conducted a study that was a comparison of six 
elements of quality between individual and conventional instruction for 37 
courses. The quality of instruction in each type of course was determined 
with a 50 question survey administered to the 1,090 students, 170 
instructors, and 54 course supervisors. A structured interview was also 


conducted at the training command of each of the 37 courses with the 
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person that was "deemed 'most knowledgeable’ of the course by one of the 

two principal investigators.” [Ref. 13: pp. 15,17] The concluding remarks 

of Braby and Evans were: 
Instructional strategy (II vs CI) {individual instruction vs 
conventional instruction] is not the determining factor in overall 
course effectiveness of current Navy courses. Rather, effectiveness 
is mediated by the extent to which good instructional practices are 
used within the courses and ultimately determined by the degree to 
which proper practices can be employed within various instructional 
strategies. Since it is possible for properly executed II and CI to be 
equally effective, a choice of strategy should rest primarily on the 
relative cost efficiencies of the two approaches. [Ref. 13: p. 33] 

There are many ways to approach a cost-effectiveness study but it 
is difficult to compare the costs of policies or programs with conclusions 
from analyses using different methods. In a 1977 Rand Corporation paper 
(Ref. 15] Marks and Massey examined several cost models used in Air Force 
programs and various case studies of non-Air Force programs. The; 
concluded that many of the studies: were inadequate as a basis for 


making "a sound resource decision," 


were lacking in the depth of analysis 
required for cost elements, and, used "poorly defined baselines against 
which to measure the cost impact of alternative cases." [Ref. 14: pp. 
§,12,14] 

In 1983, Knapp and Orlansky developed a cost element structure for 
defense training with the purpose of providing a standardized approach 
to determining the costs of a training program, regardless of size. The 
cost element structure is a listing of areas where costs should be 
measured and is relevant to a diverse range of cost studies if the analyst 


Simply disregards the cost measurement areas not applicable to his study. 


The format recommended in the study requires determining program costs 
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by measuring costs of research and development, investment costs, and 
operating costs. [Ref. 15: pp. 1,8-16] The format adds only slightly more 
detail than previous studies and doesn't provide information on how to 
obtain the costs. The study describes all costs that Knapp and Orlansky 
feel would have an effect on the budget of an organization, but a 
weakness of the approach is a resulting ignorance of the opportunity 
costs that are associated with a specific training program. 

Many studies have been conducted using a strategy similar to that 
of the cost element structure. Several have compared different military 
training methods. In 1981, Corey compared the costs of training at RM-A 
and IC-A school in an attempt to determine whether individualized 
instruction or conventional instruction was cheaper. As in the cost 
element approach, Corey's approach was to identify the resources required 
for the instruction and then to estimate the cost of the resources. The 
least cost alternative producing a given output was taken to be the most 
efficient. For this study, output was defined as the number of graduates 
trained to a certain level. All methods of instruction were assumed to 
have the same level of effectiveness. [Ref. 16: p. 13] 

Cost comparisons were made using estimated investment and operating 
costs of the training programs. The one-time investment costs included 
costs for curriculum development, specialized programming, class facilities. 
and equipment. The annual operating costs included costs of staff, 
students, hardware lease costs, facilities, maintenance, supplies, and 
curriculum maintenance costs. The staff costs used the Navy Life-Cycle 


Billet Cost Model, whic.. estimates annual costs to the government of 


23 





maintaining billets of various rates and ratings. These costs include the 
individuals' pay, allowances, and benefits. [Ref. 16: pp. 14,15] 

Using a billet cost model that includes the cost of past instructor 
training as in this study could be misleading. The past training is a sunk 
cost that was expended to develop the expertise of the instructor for his 
operational squadron and is not relevant to determining the present costs 
of training alternatives. If the required manning of the alternatives being 
compared is identical, then the added training cost will cause the total 
cost of an alternative to be overinflated, but will not affect the outcome 
of the decision among alternatives. If the manpower requirements of the 
programs differ, and alternatives are available, then the Billet Cost Model 
could influence the choice. Billet costs that include aircrew training would 
tend to become very high because of the expense of aviation training. 

The results of Corey's study show that individual instruction is less 
costly than conventional instruction over a 15 year program life. This 
result however, is sensitive to student loading and course length. When 
student loading was less than 440 students per year, conventional 
instruction was less expensive than individualized instruction. The higher 
cost of conventional instruction above the 440 students per year point is 
attributed to the longer period of time (longer than individual instruction) 
required for the student to remain in the training command. Sensitivity 
analysis suggested that if the conventionally taught RM-A course was 
shortened to eight weeks, conventional instruction was less costly. It is 
interesting to note how the student loading or course length swings the 


balance of which course is more expensive. 
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A potential weakness of the study is that for the comparisons a 
hypothetical, conventionally taught training program was used. The 
hypothesized conventional method of instruction was estimated to require 
20% more time to complete than individualized instruction. The estimate 
was based on previous studies that examined learning rates associated with 
different methods of instruction. Another possible weakness is a failure 
to use measures of opportunity cost when comparing the alternatives. 

Another cost model to determine the costs of potential changes in a 
training program was developed by Thode and Walker. Fortunately, Thode 
and Walker were involved in the ISD effort conducted at VP-31 in the late 
1970's and detailed cost information of the ISD effort and of VP-31l's 
training program are available. Their model uses a structure similar to 
the cost element structure proposed by Knapp and Orlansky. The study 
provides a working example of the model with a comprehensive analysis of 
all major costs of the P-3 training program as envisioned by the ISD 
process including aircraft, maintenance, administrative, instructor, and 
student costs. [Ref. 17: pp. 1-4] 

Lacking in the Thode and Walker study is any comparison of 
alternative teaching methods, although the model could be used to make 
such a comparison. Their model improves on other cost studies by 
including the opportunity costs associated with updating the media, and 
provides greater detail on the origin of the costs. Because of the way in 
which the cost data were aggregated in the study, it is impossible to 
determine many of the costs that would be useful for a comparison of 


mecia development and update costs. 
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The work presented here avoids the estimations required for a 
hypothesized training program because the training commands studied have 
identical training goals although they use different training media to 
achieve those goals. In addition, opportunity costs are included in the 


cost comparison of the training programs. 
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IV. METHODOLOGY 


A conceptual model is useful for categorizing and evaluating 
opportunity costs when comparing the costs of alternative instructional 
media. The conceptual model developed in this chapter is an adaptation 
of the Thode and Walker model. It categorizes costs as plant and 
equipment, development, or operating costs, and extends the Thode-Walker 
model by identifying the opportunity costs associated with each 
alternative. The model is used to compare the costs of the instructional 
media of workbooks and lectures as used in the classroom segment of the 
P-3 aircrew training programs; as stated previously the training costs 
incurred by such things as aircraft, maintenance, facilities, and 
administration, are assumed constant. The model, shown in Figure 4.1, 
divides costs into three major categories, plant and equipment, 
development costs, and operating costs for media comparison. Cost 
comparisons were cone for seven different training tracks within the Fleet 
Replacement Squadron's. The tracks were, Pilot, Naval Flight Officer, 
Acoustic Operator, Non-Acoustic Operator, Flight Engineer, In-Flight 
Technician, and Ordnanceman. 

The instructional media were determined in an early phase of the 
Instructional System Development (ISD) training program design. The 
costs to develop the selected media can vary considerably among 
alternatives. For materials development done in-house, cost estimates 


require a working knowledge of the system that produces the materials. 
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Figure 4.1. Conceptual Model to Determine Training Media Costs 


28 





For development done under contract, contracted prices should be 
available. If one medium requires substantially more organizational effort 
and labor in the production process, then higher costs will result because 
of increased manpower requirements. If the training squadron cannot 
adjust its manpower, then that medium will result in higher opportunity 
costs for the squadron because the increased administrative workload must 
generally be handled by the instructors. Areas given up by the increased 
workload are things such as instructor training time which increases 
instructor proficiency, less attention applied to other jobs held by the 
instructor, longer working hours, and, possible decreases in job 
satisfaction. The training squadrons compared in this study are manned 
equally, so if one medium requires more management effort, that increased 
effort imposes additional costs on the squadron. The additional costs may 
be poor lecture preparation, degraded instructor teaching performance, 
and lower morale if there is not enough time to accomplish all jobs 


satisfactorily. 


A. PLANT AND EQUIPMENT 

If workbook or lecture development, and later revisions, are done in- 
house, the organization must have or create a structure that can handle 
the development and revision. Each medium has different worker and 
equipment requirements necessary for development and revision. The 
purchase price of the equipment and its subsequent maintenance cost 


should be considered in comparing the costs of different media. 
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The equipment costs of the workbook method of instruction were 
determined aver discussions with the division supervisors of the 
production process and other experts in the program. All major equipment 
needed to develop or revise the workbooks were noted and prices were 
assigned based on the present purchase price of the equipment. The 
major equipment costs were computers, printers, graphics equipment, and 
printing equipment. Equipment costs of lecture presentation, ie., overhead 
projectors and slide projectors, were ignored because these items are 
required for both squadrons and are therefore not relevant to the 


comparison. 


B. DEVELOPMENT COSTS 

The development costs of the workbooks used by VP-31 were 
estimated by determining the amount of time required for the military 
subject matter experts in each track to write the books and adding it to 
an estimate of the amount of time required for all ot‘.ers in the production 
process to produce the books. Others in the process were: aircrew track 
managers, educational specialist, typists, editors, graphics artists, and 
printers. The time estimates for each production worker were based on 
a survey of the military track managers. The calculation of hourly wages 
for the subject matter experts multiplied by the production hours, summed 
over all workbooks, plus two years of worker salaries for the ISD, 
graphics, and printing departments, were used to determine the total 


workbook development cost. 
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The lecture development cost was estimated by multiplying the time 
required for an instructor to produce a lecture (obtained from instructor 
estimates) by the hourly wage of the instructor in the training track for 
which the lecture is used. The costs for all lectures in all tracks were 


summed to arrive at the total cost of lecture development. 


C. OPERATING COSTS 

Media operating costs are the costs associated with a specific training 
medium used to produce the product--the graduace. Included here are 
the costs that are a function of the graduate production process, the 
costs of keeping training media current, and costs of other attributes of 
media such as media portability and the ability of media to respond to 
changes in student loading. 

In the aviation training program, the production process is a 
conglomerate cf training activities that provides the student with the 
knowledge, skills, and abilities to do the job at the completion of training. 
The aviation training program uses a media miv that includes workbooks, 
lectures, videotapes, aircraft simulators, aircraft labs, and flights. While 
all of these media are critical to the training program, the workbooks and 
lectures teach nearly all of the theoretical knowledge in the program. 

Teaching the theoretical knowledge is a time consuming process, and 
one medium may be able to accomplish it in less time than another. Self- 
paced instruction using workbooks can theoretically reduce training time 


because students can proceed at their own pace. More recently, computer 


based instruction is making the same claim. In order to take the 











possibility of reduced training time into account the number of hours the 
students spend with the workbooks and lectures was determined from the 
training syllabi of each of the seven training tracks. 

Measuring the difference in training time alone may be misleading 
because the ability of the students upon course completion may be 
different. If a student is sped through an aircrew training program and 
sent to the fleet inadequately prepared, there is an additional cost 
imposed on the fleet squadron receiving the undertrained graduate. The 
additional cost arises from the additional time a supervisor must spend 
with the new person, the fat and maintenance incurred by remedial flight 
training, and the reduced readiness of the squadron because of 
inadequately trained people. 

There are no indications that the effectiveness of the squadrons 
compared in this study is different. Fleet squadrons on the east and west 
coasts enjoy high levels of readiness, and neither training squadron is 
receiving complaints about its training product. If the training squadrons 
are conforming to the training goals established during the ISD process, 
then they are, by definition, teaching only to the specified level of 
objectives, and the effectiveness of both squadrons should be the same. 

Along with reduced training time, reduction in manpower requirements 
brought about by the self-paced nature of the workbooks were expected 
by the developers. The manpower reduction was not realized to the extent 
originally expected. A “sterile"’ workbook environment was felt to impede 
learning because it bored the students, and in 1982, VP-31 began 


assigning instructors to student study areas, also called the learning 


32 














center, to provide assistance, answer questions, and get the students 
together for informal seminars in cases when many students were having 
difficulty with a particular area of study. This new policy increased the 
opportunity cost of the workbook method of instruction because the 
instructors now lost productive time from their other responsibilities; even 
though an instructor was on a fixed salary and was paid the same amount 
regardless of what he was doing, the instructor's productivity in his other 
assignments was decreased by added duties. 

An estimate of the costs of the production process for workbooks and 
lectures uses the student and instructor training time that is associated 
with the particular medium. It is not correct to simply apply the entire 
instructor and student salary to these estimates because a large 
proportion of the training is conducted with other media outside of the 
classroom or learning center, i.e., in ground trainers, aircraft device 
sessions, and flights. 

Determining the training time requirements for the individual media 
was done by obtaining the curriculum descriptions of the seven different 
aircrew training tracks of both training squadrons. The hours required 
for the workbook and lecture medium were determined from the curriculum 
descriptions. These hours were converted to dollars by multiplying 
instructor hourly earnings times the number of hours required for the 
particular medium, multiplying the student hourly earnings times the 
number of hours required tor the particular medium, and adding the two 


together. 
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1. Media Currency 

A continuing problem for the P-3 training program is the fact 
that the instructors must maintain training material currency. In the last 
ten years there have been seven variants of the P-3 aircraft to teach at 
the training command each with its own characteristics (P-3B mod, P-3C 
baseline, P-3C mod, P-3C UI, P-3C UII, P~3C UII.5, P-3C UIII). As a result 
the content of the training program undergoes frequent revision in an 
effort to keep up with equipment modifications to the aircraft and changes 
in the aircraft tactics that are used in the fleet environment. With 
frequently changing material, a desirable feature of any training medium 
would be the ability to make changes at relatively low cost. Lectures and 
workbooks differ substantially in this respect: major workbook revisions 
may require six months or more to go through the production system, but 
changes to lectures can usually be accomplished in several hours and the 
new material can be presented at the next lecture. 

Workbooks developed under the ISD process take up to year to 
revise because revisions to them require the efforts of many people. 
Subject matter experts must review the old workbooks for content 
currency, note areas that need revision, and write new sections to reflect 
the aircraft hardware changes. Track managers who administer the 
workbooks assigned to their tracks must note changes to the aircraft or 
software and initiate the change procedure. Educational specialists must 
take the input from the subject matter experts and make **es changes 
necessary to ensure that the workbook continues to accomplish its original 


purpose. Editors, typists, and graphic artists work to make the final 
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copy. Only then can the new material go to press, which requires the 
efforts of the people in the printing division. In contrast, an instructor 
updating his lecture may request some graphics support for updating 
transparencies or class handouts, but the normal delay is a matter of 
hours. In the case of major revisions, revising lectures may require 
several days at the most. 

The costs of maintaining media currency can be determined by 
estimating the costs to revise the media. Measures of revision costs are 
not readily available because they vary from one type of training to 
another and depend on how frequently the course content changes. 
Estimates of these costs must be obtained from the person that makes the 
changes. Even then the quoted number of hours are usually estimates of 
what is perceived as typical, and are not based on specific records. For 
this study, however, estimates of the number of revisions per year and 
the size of the revisions are based on the records kept for workbook 
revision. The revision costs of workbooks used by VP-31 were estimated 
in a manner similar to development cost estimates. 

The time estimates for each person in the revision process were 
based on a survey of the military track managers. The hourly wages of 
those involved in the changes were multiplied by the time required to 
make the changes. Total costs were obtained by summing the changes for 
all workbooks over all aircrew tracks. The wages of production workers 
other than instructors who were assigned to the ISD, graphics, printing 


departments, were included because they exist to make the revisions. The 
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annual operating budgets of these divisions were also included, and 
provide an estimate of the cost of supplies used in the process. 

The costs to revise the lectures used by VP-30 and VP-31 were 
estimated by multiplying the time required for an instructor to revise a 
lecture by the hourly salary of the instructor responsible for the lecture. 
The costs for all lectures of all tracks were summed to arrive at the total 
cost of lecture revision. Because it is difficult to determine how many 
lectures needed revision. the percent of workbooks revised was assumed 
to reflect the number of lectures needing revision. A percentage of the 
wages of VP-30 graphics people and their operating budget were included 
in the lecture revision costs for VP-30 because the graphics division 
exists in part to support their lectures. 

It could be argued that using the hourly wage of the instructors 
making the revisions is not valid because the military instructors are 
being paid regardless of how many changes they make. While that 
observation is true, a cost measurement must be made in order to compare 
the opportunity costs of one medium with another. If the instructor was 
not doing revisions, he could have been engaged in some other productive 
or useful activity, and this is in fact the cost associated with the changes. 

2. Media Portability 

Another attribute of a training medium that is often desirable is 
one of portability if training must be provided away from the usual 
training site. Within aviation commands, the aircraft are occasionally 
upgraded to a newer model. When a squadron transitions to a different 


model of aircraft, the fleet squadron must train its aircrewmen on the new 
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equipment and software. The training required for the transition can be 
done at either the Fleet Replacement Squadron by bringing all aircrewmen 
to the training site, or at the operational squadron's location. 

In the case of workbooks and lectures, both media are portable 
but transferring these incurs different costs. The workbooks can be 
bundled up and mailed off, or the instructors can catch the next plane. 
The cost of bringing training to the fleet squadron rises quickly if many 
instructors are required to conduct the training. A problem with sending 
many instructors away from their training command is that the instructors 
remaining behind must fill in for the absent instructors. Filling in for the 
absent instructors may result in a tired staff and dissatisfaction resulting 
from overwork. A training program structured around workbooks may cost 
less to transport because workbooks do not require instructors. 

A cost measure of media portability has been developed by using 
a scenario in which a four week training program is assumed to take place 
at the operational squadron's location. The assumptions are that under 
the workbook program two instructors per aircrew position are required 
along with the workbooks, and that three instructors are required to do 
the training using the lecture method. The costs are computed by 
multiplying the instructor's hourly wage times the number of hours of 
instruction, adding travel expenses and summing these amounts over all 
aircrew positions. The costs could also be determined for a scenario 
where the fleet aircrewmen report to the Fleet Replacement Squadron for 


the training. 


a1 


3. Staff Absence, Student Loading, Standardization, and Turnover 

Other cost advantages occur because some instructional programs 
can function better than others in the face of instructor absences. Navy 
training programs operate under rigid time lines with very little time 
allowed for making up missed training events. Instructor absence cannot, 
therefore, be easily accommodated. A medium such as workbooks, which 
presents the training material without the presence of an instructor, has 
a cost advantage over a lecture method that would require a replacement 
instructor to keep the training program on schedule. 

However, the production workers are critical to the revision 
process of the workbook medium. If staffing level is at the level required 
for the useful flow of changes, someone's absence for a lengthy period or 
a job unfilled because of a hiring freeze, can lead to a large revision 
backlog and long periods with inaccurate workbooks. The lecture medium 
requires few production workers, and those do not hold critical positions, 
so their absence has little affect on a lecture program. 

Some training programs may also be faced with surges or declines 
in student loading. The workbook medium can easily deal with a 
temporary surge in student loading as long as the books are available. 
Printing extra workbooks is relatively inexpensive and should not require 
too much lead time. With lectures, when increased student loading exceeds 
the class capacity additional instructors are required. Because instructors 
have numerous other responsibilities in the training command, the added 
demand imposes a cost because of the reduced productivity in other areas 


of responsibility. Student loading in P-3 training is typically stable so an 
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operating cost resulting from changes in student loading was not 
determined, however, in a qualitative sense workbooks should have a cost 
advantage if students increase or decrease. 

The ease with which a training program can be standardized will 
vary from one program to another depending on the media used in the 
training program among other things. Poor standardization in the training 
program may lead to a failure to meet the objectives for which the 
training program was established. The gap between what a squadron is 
really teaching and what should be taught as determined by the training 
objectives has an opportunity cost associated with it. That cost is 
manifested in the extra effort required by the fleet squadron to bring the 
graduate to the level he should have been upon reporting. 

People are probably the source of the greatest variability in 
training courses. Student learning can vary greatly depending on 
instructor knowledge, instructor teaching ability, and student classroom 
attention. Instructor knowledge and teaching ability can vary greatly 
from one instructor to another. Some instructors are far better at 
recognizing when a student is having difficulty, and at figuring out other 
ways to present the material. Students have their good and bad days, 
and their attention in the class is not always the same. Any material 
presented by the instructor during a student's attention lapse is lost to 
the student. 

AR workbook presents information the same way every time. This 
reduces variability due to the instructor. With a workbook, a student may 


continuously gauge his performance relative to the exact requirements of 


Se 


the course using the quizzes provided. Poor performance in certain areas 
prompts a student to go back and study the areas that were not learned 
well enough. Students completing the workbooks ahead of schedule may 
have the opportunity to expand their knowledge of advanced material. An 
attention lapse does not have the same affect as in the lectures because 
the student still has the course material in front of him. By design, the 
workbook method with its fixed presentation and more structured change 
mechanism, is better standardized than lectures. 

Staff turnover is a problem incurred regardless of the media 
used for instruction, but there may be differences in the media that makes 
one less costly in terms of staff turnover than the other. With rapid staff 
turnover, an instructor is often transferred shortly after developing his 
expertise. A longer time in a position is usually accompanied by a greater 
proficiency within the position. Since aircrew training is accomplished 
with military instructors, rapid turnover is common as people routinely 
leave the training organization after several years of duty. 

Instructors involved with a lecture method of instruction spend 
several weeks or more learning to present lectures, an activity not 
required for a workbook method of instruction. However, the greatest 
cost arising from rapid turnover in an ISD developed training program 
may be from the changes that are made to the training program. People 
with experience and an in-depth understanding of the ISD process from 
which the training program was developed are rare. Someone without 
sufficient knowledge may make changes to the training program, the 


workbooks, or the lectures, that are not consistent with the objectivec of 
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the training program. The cost of rapid turnover may then depend on 
how much damage was done by those who made the incorrect or 
unnecessary changes. In the P-3 training program, the workbooks require 
less instructor preparation time, but damaging changes to the training 
program could be made by the staff in either method of instruction. 

There may be other attributes that are important to evaluate 
when considering different training media. When evaluating media 
alternatives, conventional instruction is usually thought of as the baseline 
in the comparison. Instructor or student prejudice may make the use of 
an alternative difficult. A teaching method may not be readily accepted 
by either the instructors or the students because it is different or 
unfamiliar. The lecture method of teaching is something that all students 
grew up with and usually expect when entering the training command. 
The instructcrs may have a natural bias towards lectures because they 
envision lectures as the proper way to teach and are unfamiliar with other 
methods. 

In aviation training, most instructional hours are devoted to 
teaching technical material. It is a very tedious process to learn the 
material with a static medium such as workbooks. Learning through 
lectures seems to ease this burden. As mentioned earlier, this was the 
reason VP-31 instituted a learning center instructor in 1982. This 
indicates that the costs of a workbook program must also include the costs 


of instructor interaction required to maintain similar levels of training. 
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V. DATA AND ANALYSIS 


This chapter compares the costs of the workbook and lecture training 
programs in use at VP~31 and VP-30. Tables 5.13, 5.14, and 5.15 present 
the costs for the comparison in the same format as the cost model 
described in chapter four, and the other tables in this chapter show how 
those costs were derived. 

Table 5.1 shows the approximate annual student enrollment for the 
different aircrew positions. The student loading was determined by 
calculating the fleet squadron's manpower demands for the twelve 
operational squadrons supported by its Fleet Replacement Squadron. The 
calculations were based on each sgvadron requiring three pilots, two naval 
flight officers, two flight engineers, two acoustic operators, one non- 
acoustic operator, one ordnanceman, and one inflight technician per P-3 
crew. Each operational squadron was assumed to require ten crews. The 
length of tour used in the calculation was three years for flight engineer, 
ordnanceman, and inflight technician, and three and a half years for all 
others. 

The military salaries used in these computations are shown in Table 
5.2 and were obtained from NAVCOMPTNOTE 7041 of 8 NOV 1989 which lists 
the composite standard rates for military people. These rates are 
prescribed for use when the costs of military services are determined on 


a statistical basis. Civilian salaries were obtained from the general 
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TABLE 5.1. FLEET REPLACEMENT SQUADRON STUDENT LOADING 


Aircrew Position Annual Number of Students 





Pilot 120 
Naval Flight Officer 80 
Flight Engineer 80 
Acoustic Operator 69 
Non~-Acoustic Operator 35 
Ordnanceman 40 
Inflight Technician 40 





schedule salary table number 75, authorized by Executive Order 12698 of 
December 23, 1989, and were multiplied by a fringe benefit factor of 1.184. 
The addition of the fringe benefit factor provides a better estimate of the 
actual costs of employing civilian workers. For military and civilian 
workers, the hourly wage was based on 2080 working hours per year as 


specified in the NAVCOMPT manual. 


TABLE 5.2. MILITARY AND CIVILIAN WAGE RATES 
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Rank/Rate/Grade Annual Cost Hourly Cost 
03 64,336 $30.93 
02 49,450 23.77 
O1 37,394 17.98 
E~-7 44,152 21.23 
E-6 37,085 17.83 
E-5 30,760 14.79 
E-4 25,530 12.27 
E-3 21,621 10.39 
GS-12 43,830 21.07 
GS-11 36,570 17.58 
GS-9 30,226 14.53 
GS-7 24,708 11.88 
GS-5 19,949 9.59 
GS-4 17,830 8.57 
GS-3 15,883 7.64 


Table 5.3 provides estimates of the hours per course spent by 
students and instructors in the two instrutional programs. The estimates 
were arrived at using the curriculum descriptions for each of the seven 
aircrew training tracks at the Fleet Replacement Squadrons. For VP-31, 
the student learning center hours represent the scheduled hours that the 
students spent studying the workbooks and taking ‘he unit tests. At 
VP-31, an instructor is required to be present i: the learning center 
during the time that students are assigned there. The reason that the 
instructor learning center hours are less than the student hours is 
because of an overlap in classes. For example, four different classes of 
Naval Flight Officers are at different stages of VP-3l1's training program 
simultaneously. If students from each of the four classes are scheduled 
for the learning center on the same day, only one instructor is required 
for all four classes. Learning center usage at VP~-30 is mostly for self- 
study of reference materials other than workbooks, and instructors are not 
usually assigned. However, instructors are still required to administer 
tests. The stucent's rank or rate which is needed for later cost 
comparisc .S, is indicated in parenthesis. 

The hours shown for flight engineer, ordnanceman, and inflight 
technician do not represent the total hours of training which these aircrew 
positions receive prior to graduation. The training program for these 
positions is divided among different commands making it difficult to 
compare training hours with the given media. The hours shown under 
VP~-31 are the hours spent in lectures or the learning center at VP-31. 
The training program at VP-30, for the same aircrew positions, was so 


different that a direct comparison could not be made. VP-3l's training 
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TABLE 5.3. LEARNING CENTER AND LECTURE HOURS PER COURSE 
AT VP-31 AND VP-30 














VP-31 VP-30 
Aircrew Position Learning Cntr, Lecture Learning Cntr, Lecture 
Student Inst. Student Inst. 
Pilot (01) 400 134 75 249 18 126 
Naval Fight Off.(01) 258 86 114 228 27 186 
Flight Eng. (E-5) 88 16 224 88 16 224 
Acoustic (E-4) 239 80 219 200 65 166 
Non-Acoustic (E-4) 275 92 282 175 60 288 
Ordnanceman (E-3) 2 0 88 2 0 88 
Inflight Tech. (E-4) 28 20 19 28 20 19 





hours were used on both sides of the equation which provided the ability 
to count the hours associated with the workbooks at VP-31, and later, the 
costs, The training conducted at VP-30 was assumed to be conducted with 
lectures. Because the flight engineer and ordnanceman training tracks at 
VP-31 use more lectures than workbooks, this assumption could bias the 
study. If the lecture method is mo.v expensive, the assumption increases 
the costs of VP-31's training, if the lecture method is less expensive, it 
will cause VP-3l1's costs to be lower. 

Annual hours for each traini.g program were computed by multiplying 
the hours per course by the number of courses annually. VP-31 operates 
its training program using a 24 day schedule. Each 24 days is called a 
phase. Most of the students in the different aircrew tracks require four 
phases, or 96 training days, to complete their training. At VP-31, classes 
begin after the rotation of a phase, or every 24 days. VP-30's training 
program is 90 training days long and is divided into three phases, each 


phase being 30 days long. Dividing the trainir.: program into three or 








four phases means that VP-31, with the four phase system, teaches ten 
new classes per year. VP-30 with the three phase system teaches eight 
new classes per year. The number of classes per year is needed for later 
calculations. 

Since the number of students is identical in both training programs, 
operating with fewer classes each year, as done at VP-30, increases the 
number of students in each class. An interesting result of having eight 
instead of ten classes annually is the reduced instructor hours. For 
example, ten pilot classes at VP-31 will r~quire 2,09. instructor hours 
annually (learning center instructor plus lecture instructor times ten). If 
VP-31 changed its schedule to a -hree phase system with each phase being 
32 days long, just under eight classes could be taught annually requiring 
roughly 1,672 pilot instructor hours, or 418 less hours. Changing to a 
three phase system would save 818 officer and 2,080 enlisted instructor 
hours annually over all training tracks. Changing from ten to eight 
classes annually wi!’ change the pilot class size from 12 to 15 students and 
the naval flight officer class size from eight to ten. Changes of this 
magnitude are negligible, and should not affect the quality of training or 
the instructor effort required to teach the class. Larger differences, 
caused by eliminating one of the Fleet Replacement Squadrons, would 
increase class size to around thirty. Teaching thirty students would 
create a sionificantly heavier demanc on instructor time, and create new 
problems not seen with the change from 12 to 15. One cost of changing 
to a three phase system is the loss in productivity of the students 


because the students will be delayed in reporting to their operational 
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squadrons by the same amount of delay in starting the training program. 
Another problem is that the program set up with eight courses per year 
must schedule more aircraft trainers and aircraft flights during the flight 
phase of instruction to accommodate the increased number of students. 
Thus, while a change from 12 to 15 students may be negligible, the same 
cannot be said of an increase from 12 to 30, 

In order to determine the cost of media development and the cost of 
maintaining media currency, an estimate of the time it takes to develop or 
revise a workbook or lecture is needed. A survey completed by several 
VP-31 track managers and discussion with the VP-31 educational specialist 
provided the information used to make these estimates. Table 5.4 shows 
the time needed for major or minor revisions to workbooks and the time 
taken annually for workbook review averaged across all seven training 
tracks. Major revisions required because of substantial equipment changes 
usually require a complete remake of the workbook. Minor revisions may 
require up to several pages of changes. An annual review is the time 
spent reviewing a workbook without having to make changes. Roughly 33 
percent of the 850 workbooks in the system require major revisions 
annually and 66 percent require minor revisions. Twenty two percent of 
the workbooks are reviewed annually with no changes. These percentages 
exceed 100 percent because some workbooks require more than one 
revision annually. The total hours shown represent the hours required 
for all training tracks and come from a mix of different wage earners in 


the revision process. A breakdown of the hours required for the different 
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functions in the revision process is shown to provide an idea of the effort 
involved in workbook revisions. 

The annual cost for revisions is shown in Table 5.5. Computing this 
cost required determining the wages of all people in the revision process, 
determining the hours required to make the revision, and determining how 
many revisions are made within the different training tracks. 

The revision costs associated with subject matter experts and track 
managers were calculated as follows. A total of 850 workbooks are 
required in the training program to teach the information associated with 
ali the different models of P-3's. Of these, 350 are used in the pilot and 
naval flight officer tracks. For major revisions in the officer tracks, the 
350 workbooks were multiphed by 33.67 percent (the number of major 
revisions annually). This number was multiplied by the average hours 
reguired for major revisions (27 hours) and by the hourly wage rate for 
officers at the C3 level ($30.93). The computation for the enlisted tracks 
was done in a similar manner. The remaining 500 enlisted workbooks (850 
total workbooks - 350 officer workbooks) was multiplied by 33.67 percent, 
by the revision time of 27 hours, and by the E-€ wage race of $17.83. 
The officer and enlisted revision costs were then added tojether. 

Production workers in the ISD, graphics, and printing departments, 
are employed to keep the workbooks current. Because these departments 
engage in some activities not related to worxbook revision, only a 
percentage of their annual salacies, determined during supervisor 


interviews, were used. One hundred percent of the ISD salaries, 60% of 
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VP-3l's graphics salaries, and 90% of the salaries of the graphics and 
printing departments under the Fleet Aviation Specialized Operational 
Training Group (FASO) were used. These salary percentages were divided 
between the major and minor revisions based on the estimated effort 


required for each type of revision. 


TABLE 5.4. TIME REQUIRED FOR MAJOR AND MINOR REVISIONS, 
AND ANNUAL REVIEW OF WORKBOOKS 








Average Total Hours 

Position Hours/Workboaok Per /Year 
MAJOR REVISION 

Subject Matter Expert 27.0 8182 
Track Manager 18.8 5707 
Educational Specialist 4,0 1212 
Editor 4.0 7212 
Typists 16.0 4848 
Graphics 8.5 257€ 
Print 2.0 606 
MINOR REVISION 

Subject Matter Expert 3.8 2202 
Track Manager 4.8 2777 
Educational Specialist 2.0 1149 
Editor 15 862 
Typists 7.0 4021 
Graphics 4.2 2394 
Print 2.0 1149 
ANNUAL REVIEW 

Subject Matter Expert ay, 340 
Track Manager 2.0 408 











TABLE 5.5. COSTS FOR MAJOR AND MINOR REVISIONS, 
AND ANNUAL REVIEW OF WORKBOOKS 








Cost For 
Position Revisions 
MAJOR REVISION 
Subject Matter Expert $179,460 
Track Manager 124,960 
ISD Department 84,750 
Graphics 71,340 
Print 48,000 


Sub Total $508,510 
MINOR REVISION 


Subject Matter Expert $47,890 
Track Manager 60,490 
ISD Department 72,190 
Graphics 60,770 
Print 70,080 


Sub Total $311,420 
ANNUAL REVIEW 


Subject Matter Expert $7610 
Track Manager 8950 


Sub Total $16,560 


TOTAL $837,310 





Original workbook development was a major undertaking during the 
instructional system development (ISD) process. To estimate the workbook 
development costs shown in Table 5.6, the development hours per workbook 
for the subject matter experts and track managers were assumed to be the 
same as the hours required for a major workbook revision. The 
development cost estimate is based on developing 850 workbooks, each of 


which has 35 pages. The computations are identical to the computations 











for major workbook revision, except that 100 percent of the workbooks 
underwent development. These hours, and the costs determined from the 
hours, probably underestimate actual development time and cost because 
a development process usually requires many more iterations to produce 
the print ready-copy than a revision process because the workbook form 
has already been decided. When making revisions, less effort is required 
to organize the material for orderly presentation. 

The development cost estimate for ISD, graphics, and printing, cannot 
be computed by using a percentage of annual costs as done for revision 
costs, because more than one year would be required for development. 
Here, two years of total annual salaries for ISD, graphics, and printing are 
assumed. A two year cost estimate still probably underestimates these 
development costs. The actual original curriculum development costs were 
$7,962,000 as presented by the Thode-Walker study [Ref. 17]. But, these 
costs represent the development costs for the entire curriculum and 


include developing other course media such as videotapes and aircraft 





events. 
TABLE 5.6. COST OF WORKBOOK DEVELOPMENT 
SE a I I I A RS NET SNCS I NT SN SY EP EI IONS TS DS SSS A a TT SS Ie 
Total Development Total Development 

Position Time (Hrs) Cost 
Subject Matter Expert 22,950 $532,900 
Track Manager 15,380 371,060 
ISD Department 20,400 313,870 
Graphics 7,225 370,240 
Print 1,700 262,410 


TOTAL $1,850,480 
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Lecture development and revision time was more difficult to measure 
because these times vary considerably depending on the technical nature 
of the lecture and the expertise of the person developing it. From 
discussions with instructors and responses to the author's survey, 12 
hours was chosen as the estimate for the time required to develop one 
hour of lecture. This same time was used as an estimate of the time 
required for a major revision for one hour of lecture. Two hours of 
revision time was used as the time required for a minor revision. The 
revision frequency was assumed to be that used for the workbooks, Le., 
33% of the lectures had major revisions annually and 68% had minor 
revisions annually. The time and cost of lecture development and revision 


are shown in Tables 5.7 and 5.8. 


TABLE 5.7. COST OF LECTURE DEVELOPMENT 








VP-31 VP-30 
Development Development Development Development 

Position Time (Hrs) Cost Time (Hrs) Cost 

Pilot 900 $27,840 1520 $47,020 
Naval Flight off. 1368 42,320 2240 69,290 
Flight Eng. 2688 47,930 2688 47,930 
Acoustic 2628 46,850 1992 35,520 
Non-Acoustic 3384 60,340 3456 61,620 
Ordnanceman 1056 18,830 1056 18,830 
Inflight Tech. 228 4,070 228 4,070 
TOTAL $248,180 $284,280 








TABLE 5.8. COSTS OF LECTURE REVISIONS 
BY TRAINING TRACK 








Major Revision Minor Revision 
Instructor Hours Dollars Hours Dollars 
VP-31 
Pilot (03) 297 $9,190 102 $3,160 
Naval Flight Off. (03) 451 13,950 155 4,800 
Flight Eng. (E-6) 887 15,820 304 5,420 
Acoustic (E-6) 867 15,460 297 5,300 
Non-Acoustic (E-6) 1116 19,900 383 6,830 
Ordnanceman (E-6) 348 6,210 119 2,130 
Inflight Tech. (E-6) 75 1,340 25 450 
TOTAL $81,870 $28,090 
VP-30 
Pilot (03) 499 $15,440 171 $5,290 
Naval Flight Off. (03) 736 22,770 253 7,830 
Flight Eng. (E-6) 887 15,820 304 5,420 
Acoustic (E-6) 657 11,720 225 4,020 
Non~-Acoustic (E-6) 1140 20,330 391 6,980 
Ordnanceman (£-6) 348 6,210 119 2,130 
Inflight Tech. (E-6) 75 1,340 25 450 
TOTAL $93,630 $32,120 





The graphics and printing departments exist to support revisions to 
the workbook media so their operating budgets were ~° luded in the cost 
estimate for media currency and are shown in Table 5.9. Because some of 
their functions are not workbook related, only ninety percent of FASO's 
graphics and printing department budgets were used. VP-3l's graphics 
department budget was not included because it would exist even if the 


workbooks were not in the training program, just as VP-30's graphics 


department exists. The percentages were determined through interviews 


with the respective supervisors. 


TABLE 5.9. ANNUAL BUDGET OF DEPARTMENTS 
ASSOCIATED WITH WORKBOOKS 








Department Annual Dollars 
Graphics $11800 
Print 9720 
TOTAL $21,520 





The labor cost of each course, which was estimated as the time that 
the instructors and students are involved with the media, is shown in 
Table 5.10 and 5.11. The costs shown in the these tables were determined 
by multiplying the wage rate for the instructors and students by the 


hours spent with the media as shown earlier in Table 5.3. 


TABLE 5.10. COSTS PER COURSE FOR INSTRUCTOR TIME WITH MEDIA 











VP-31 VP-30 

Instructor LCx LECT* LC LECT 
Pilot $4150 $2320 $560 $3900 
Naval Flight Off. 2660 3530 840 5760 
Flight Eng. 290 4000 290 4000 
Acoustic 1430 3910 1160 2960 
Non-Acoustic 1640 5030 1070 5140 
Ordnanceman 0 1570 0 1570 
Inflight Tech. 360 340 360 340 
Sub Total $10,530 $20,700 $4,280 $23,670 
TOTALS (LC+LECT) $31,230 $27,950 





* LC stands fer learning center, LECT stands for lecture. 
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TABLE 5.11. COSTS PER COURSE FOR STUDENT'S TIME WITH MEDIA 
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VP-31 VP~-30 

Instructor LC LECT LC LECT 

Pilot $86,310 $16,190 $67,160 $33,990 
Naval Flight Off. 37,120 16,400 41,000 33,450 
Flight Eng. 10,420 26,510 13,620 33,130 
Acoustic 20,240 18,550 21,170 17,570 
Non-~Acoustic 11,810 12,120 9,400 15,460 
Ordnanceman 90 3,660 110 4,580 
Inflight Tech. 1,380 940 1,720 1,170 
Sub Total $167,370 $94,370 $153,580 $139,350 
TOTALS (LC+tLECT) $261,740 $292,930 





The cost of portability, shown in Table 5.12, was determined using 
a scenario that assumed instruction was required at an out-of-town 
training site. For this scenario, the workbook method of instruction was 
assumed to require two instructors per aircrew position and the lecture 
method to require three. Under these assumptions, four officer and ten 
enlisted instructors were required to teach with the workbook method, 
while six officer and 15 enlisted were required for the lecture method. 
Training was assumed to take one month, with five working days per week, 
or a total of 160 training hours per instructor. The training time was 
multiplied by the instructors’ wages. A daily per diem rate of $24 per 


day per instructor was added. 
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TABLE 5.12. COST OF PORTABILITY 








VP-31 VP-30 
No. of Instructors Dollars No. of Instructors Dollars 
4 Officers (03) $22,680 6 Officers (03) $34,020 
10 Enlisted (E-6) 35,730 15 Enlisted (E-6) 53,600 
TOTAL $58,410 $87,620 





The plant and equipment costs for VP-3l1 are shown in Table 5.13. 
These costs are the purchase costs of the eculpment needed to support 
changes to the workbooks and are measured in current dollars. Equipment 
costs to support changes to lectures wer? assumed to be negligible. 

Table 5.14 shows the combined opportunity costs estimate associated 
with developing the workbooks and lectures at VP~3l and the lectures at 
VP-30. The development costs for lectures at VP-31 and VP-30 are nearly 
the same which indicates that a large number of lectures have been 
instituted at VP-31. The high worsbook development cost is the result of 
the great amount of time required to develop textbooks. 

Development costs are sunk costs and of importance only if a new 
development effort is contemplated. Table 5.15 compares the present 
Operating costs based on the opportunity costs associated with the actions 
required to support either medium. The production process costs shown 
are fairly similar, indicating that both training programs spend about the 
Same amount of time presenting the theoretical information. The large 
difference in the currency costs is due to the effort required of the 


Subject matter experts and track managers, and the extra manpower 











required in the ISD, graphics, and printing departments, to revise the 
workbooks. The cost differences in portability, as shown in Taple 5.15, 
indicate that the workbooks have a portability advantage. This portability 
advantage results from the fact that workbooks can substitute for 


instructors. 


TABLE 5.13. PLANT AND EQUIPMENT COSTS AT VP-31 








Department Equipment Cost 
ISD $37,575 
Graphics 45,700 
Printing 57,600 
TOTAL $140,875 





TABLE 5.14. DEVELOPMENT COSTS 








Media VP-31 VP-30 

Lectures $248,180 $284,280 
Workbooks 1,850,480 0 
TOTAL $2,098,660 $284,280 





TABLE 5.15. ANNUAL OPERATING COSTS 








Measure VP-31 VP-30 

Production Process $2,929,700 $2,567,040 
Currency 968,790 125,750 
Portability 29,210 43,810 
TOTAL $3,927,700 $2,736,600 





The costs from Tables 5.13, 5.14, and 5.15, were used to determine the 
life-cycle costs of the training programs over a ten year period. These 
costs are shown in Table 5.16. To compute the life-cycle costs, 50% of 
the plant and equipment costs were assumed to depreciate to zero and 
need replacing at the end of ten years. The replacement costs of plant 
and equipment were assumed to increase at an annual inflation rate of 4%. 
Development costs were assumed to be a one time cost with no replacement 
costs. The annual operating costs are composed mostly of salaries, and 
these salaries were assumed to increase at an annual rate of 3%. For the 
net present value comp. ations, a discount rate of 9% was used. To 
account for inflation, the discount rate was combined with the inflation 
rate using the formula 


nominal rate (i) = real rate (r) + inflation rate (x) + ra. 


TABLE 5.16. TEN YEAR LIFE-CYCLE COSTS 





Ten Year Costs 





Cost Category VP-31 VP-30 
Plant and Equipment 

ISD $45,510 sO 

Graphics 55,360 0 

Printing 69,770 0 
Development Costs 

Lectures 248,180 284,280 

Workbooks 1,850,480 0 
Production Process 17,433,770 15,275,690 
Currency 5,764,980 748,300 
Portability 173,820 260,700 
TOTAL $25,641,870 $16,568,970 











Scheduled aircraft transitions are expected to reduce the number of 
P-3 aircraft models used at operational squadrons from seven to three by 
1992, The reduced number of models could reduce the number of 
workbooks from roughly 850 to 550. A substantial savings in opportunity 
cost will be realized with fewer workbooks because of reduced revision 
costs. Further savings in workbook revision will result from a workbook 
design change presently in progress. The design change modifies the 
workbooks so that frequently changing material will be removed and the 
students will be referred to reference manuals that are identical to the 
manuals used the fleet squadrons. With this new system, a 90% revision 
reduction time is estimated for the subject matter experts and track 
managers. The costs incurred by the ISD, graphics, and printing 
departments, will not decline with a reduction in the number of workbooks 
or revisions because nearly all of their costs are assigned to workbook 
revision. Their total costs will remain the same because ISD, graphics, and 
printing are now staffed at a minimum functional level so that further 
reductions are not likely. The cost of maintaining workbook currency with 
fewer workbooks and the new workbook design is shown in Table 5.17, 
The “new system.” cost also includes the operating budgets from Table 5.9 
as before. Tables 5.18 and 5.19 show the costs of workbook and lecture 


revisions that were consolidated in Table 5.17. 








TABLE 5.17. THE CHANGE IN CURRENCY COST WITH 
FEWER WORKBOOKS AND LECTURES 








Measure VP-31 VP-30 
Old Currency $ $968,790 $125,750 
New system 528,320 81,74. 
Difference $440,450 (45% less) $44,010 (35% less) 





TABLE 5.18. COSTS FOR MAJOR AND MINOR REVISIONS, 
AND ANNUAL REVIEW OF WORKBOOKS* 








Cost For 
Position Revisions 
MAJOR REVISION 
Subject Matter Expert $11,670 
Track Manager 8,140 
ISD Department 84,750 
Graphics 71,340 
Print 48,000 


Sub Total $223,900 
MINOR REVISION 


Subject Matter Expert $3,140 
Track Manager 3,960 
ISD Department 72,190 
Graphics 60,770 
Print 70,080 


Sub Total $210,140 
ANNUAL REVIEW 


Subject Matter Expert $690 
Track Manager 590 


Sub Total $1,280 


TOTAL $435,320 





* Based on 550 workbooks and 90% less revision time for 
subject matter experts and track managers. 
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TABLE 5.19. COSTS OF LECTURE REVISIONS 
BY TRAINING TRACK* 








Major Revision Minor Revision 
Instructor Hours Dollars Hours Dollars 
VP-31 
Pilot (03) 297 $9,190 102 $3,160 
Naval Flight Off. (03) 451 13,950 155 4,800 
Flight Eng. (E-6) 887 15,820 304 5,420 
Acoustic (E-6) 867 15,460 297 5,300 
Non-Acoustic (E~-6) 1116 19,900 383 6,830 
Ordnanceman (E-6) 348 6,210 119 2,130 
Inflight Tech. (E-6) 75 1,340 25 450 
TOTAL $53,220 $18,260 
VP-30 
Pilot (03) 499 $15,440 272 $5,290 
Naval Flight Off. (03) 736 22,770 253 7,830 
Flight Eng. (E-6) 887 15,820 304 5,420 
Acoustic (E-6) 657 11,720 225 4,020 
Non-Acoustic (E-6) 1140 20,330 391 6,980 
Ordnanceman (E-6) 348 6,210 119 2,130 
Inflight Tech. (E-6) 75 1,340 25 450 
SOTRG $60,860 $20,880 


* Based on a 35% reduction in the number of lectures that require 
revision brought about by reducing the number of aircraft models used 
in the fleet. 


The Navy is faced with force cutbacks and base closures which could 
affect the number of students in the Fleet Replacement Squadrons. Table 
5.20 shows the annual demand for students under two different scenarios. 
The numbers under "fewer" are the annual number of students that would 


require training after cutting one P-3 squadron from each coast and 


reducing the number of crews per squadron to nine. The numbers under 





“more” are the number of students that would require training if the 
same cuts anticipated in the first scenario were made and if aircrew 


training was consolidated at either VP-3l or VP-30. 


TABLE 5.20. FLEET REPLACEMENT SQUADRON STUDENT LOADING 





Number of Students 





Aircrew Position Status Quo Fewer More 
Pilot 120 99 198 
Naval Flight Officer 80 66 132 
Flight Engineer 80 66 132 
Acoustic Operator 69 57 114 
Non-Acoustic Operator 35 28 56 
Ordnanceman 40 33 66 
Inflight Technician 40 33 66 


The variability of production costs over a range of student loading, 
shown in Table 5.21, was estimated using the number of students displayed 
in Table 5.20. The costs of the instructor time spent with the media were 
assumed to stay the same as those computed in Table 5.10. Student costs 
were computed in the same manner as earlier. The costs shown in Tables 
5.22 and 5,23 are the costs that were used to derive these production 
costs. The production cost, which was defined as the time the students 
and instructors spend with the training medium, is higher for VP-3l over 
all ranges of students. As student loading increases, the costs for both 


methods increases at the same rate. 








TABLE 5.21. THE CHANGE IN PRODUCTION COST WITH 
CHANGES IN STUDENT LOADING 
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Measure Workbooks Lectures 
Status Quo $2,929,700 $2,567,040 
Fewer Students 2,465,800 2,152,400 
More Students 4,619,000 4,080,480 





TABLE 5.22. COSTS PER COURSE FOR STUDENTS TIME WITH MEDIA* 





VP-31 VP-30 

Instructor LC LECT LC LECT 

Pylot $71,200 $13,350 $55,410 $28,040 
Naval Flight Off. 30,620 13,530 33,820 27,590 
Flight Eng. 8,600 21,870 10,740 27,340 
Acoustic 16,720 15,320 17,4390 14,520 
Non-Acoustic 9,450 9,690 7,520 12,370 
Ordnanceman 70 3,020 90 3,780 
Inflight Tech. 1,140 770 1,420 970 
Sub Total $137,800 $77,550 $126,490 $114,610 
TOTALS (LC+LEcCT) $215,350 $241,100 





x Based on 11 squadrons per coast and nine crews per squadron. 


If the workbook training method had worked as originally designed, 
lectures and learning center instructors would not have been required. 
The learning center hours shown in Table 5.24 under VP-31 are the 
learning center hours prescribed by the original ISD design. As an 


estimate of the opportunity costs incurred because of these changes, the 





TABLE 5.23. COSTS PER COURSE FOR STUDENTS TIME WITH MEDIA* 








VP-31 VP-30 

Ir structor LC LECT LC LECT 
Pilot $142,410 $26,700 $110,810 $56,070 

Naval Flight Off. 61,240 27,060 67,640 55,180 ’ 
Flight Eng. 17,180 43,740 21,480 54,670 

Acoustic 33,430 30,640 34,970 29,030 

Non-Acoustic 18,900 19,380 15,030 24,740 

Ordnanceman 140 6,040 170 7,550 

Inflight Tech. 2,210 1,540 2,840 1,930 

Sub Total $275,570 $155,100 $252,940 $229,170 

TOTALS (LC+LECT) $430,670 $482,110 





* Based on 22 squadrons and nine crews per squadron. 


cost of the originally designed method was estimated and is shown in Table 
5.25. The table also compares the present currency cost of workbooks 
only, and the 190 Thode-Walker estimate [Ref. 17] of maintaining workbook 
currency. The present workbook currency cost was obtained by . 
subtracting the costs required to maintain lecture currency from the 
currency cost in Table 5.15. The developer estimate was obtained by 
applying current salaries to the computations used in the Thode-Walker 
study, and adding the graphics and printing department costs that weve 


not included in their study. 
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TABLE 5.24. LEARNING CENTER AND LECTURE HOURS PER COURSE 
AT VP-31 AND VP-30* 








VP-31 VP~30 
Aircrew Position Learning Cntr, Lecture Learning Cntr, Lecture 
Student Inst. Student Inst. 
Pilot (O01) 248 18 0 249 18 126 
Naval Flight Off.(01) 386 2] 0 228 27 186 
Flight Eng. (E-5) 265 16 0 88 16 224 
Acoustic (E-4) 229 65 0 200 65 166 
Non-Acoustic (E~-4) 154 60 0 175 60 288 
Ordnanceman (E-3) 40 0 0 2 0 88 
Inflight Tech. (E-4) 28 20 0 28 20 19 





x The student hours listed under VP-31 were determined during the ISD 
process used to design the original training program. 


TABLE 5.25. PRESENT ANNUAL COSTS COMPARED WITH COSTS 
ESTIMATED AT INCEPTION 








Measure VP-31 Now Developer Est. 
Production Process $2,929,700 $1,736,860 
Currency 858,830 582,990 
TOTAL $3,788,530 $2,319,850* 





* Includes graphics and printing department costs which were not 
included in the Thode-Walker study. 
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VI. SUMMARY AND CONCLUSIONS 


This study compares two methods of instruction, workbooks and 
lectures, as presently used in P-3 Fleet Replacement Squadrons, to 
determine which method of instruction is more efficient. The study 
compares the training program at VP-31 that uses workbooks and lectures 
in a nearly one to one ratio, with the training program at VP-30 that 
supplements lectures with only a few workbooks. It is again important to 
emphasize that this study does not look at the training costs incurred by 
such things as aircraft, maintenance, facilities, and administration costs. 

The lecture training method at VP-30 is much more efficient than the 
workbook-lecture method at VP-31 based on the operating costs presented 
in Table 5.15, assuming equal effectiveness of both training systems. As 
seen in Table 5.15, tne total annual opportunity costs associated with 
teaching the majority of the theoretical information through lectures, are 
70% of the costs of the workbook and lecture method used at VP-3l. 

The higher costs of the workbook-lecture method are due to higher 
student production and revision costs. Production costs at VP~-3l are 
higher primarily because of the additional time instructors are assigned 
to the learning centers; a requirement which was unanticipated when 
selecting the workbook medium. Revision costs are higher at VP-31l 
because of the extensive effort required to keep the workbook medium 


current as new software, equipment, and tactics, enter the fleet. 
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Production costs were estimated for a range of student loadings that 
could result from two possible scenarios; reduced student loading from 
lowered fleet demand because of fleet squadron reductions, and increased 
student loading caused by closing one of the Fleet Replacement Squadrons. 
In both cases, the lecture method was still found to be the most efficient 
method of instruction. 

The cost of maintaining media currency may decrease for both 
training commands in the future. Substantially fewer revisions to 
workbooks and lectures are expected as a result of a reduced number of 
P-3 aircraft models, and as a result of a new workbook design that 
removes the most frequently changing information. VP-3l's training 
program wili benefit the most from fewer revisions but the currency cost 
at VP-31 will remain considerably higher than VP-30. The higher currency 
cost under this scenario is due primarily to the manpower requirements 
of the workbook revision process. 

The workbook-lecture method usea at VP-31 does have a lower 
portability cost. Even though the immediate need for portability appears 
to be declining because the changes to fleet squadron aircraft should be 
completed by 1992, new aircraft models will follow, and the portability of 
a medium will aiways be a consideration. 

Certain qualitative attributes should also be considered when 
evaluating the efficiency and effectiveness of training programs. The 
workbook method appears to have an advantage in its ability to contend 
with instructor absences and its ability to maintain instructional material 


standardization although it can be hindered by production worker absence. 
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If the workbook method of instruction had been maintained exactly as 
designed by the Instructional Systems Development (ISD) program, t’.e 
workbook method would now cost much less than VP-31's present program, 
and only 87% of the lecture method now used at VP-30. At VP-31, the 
production process costs are much higher than anticipated because of the 
necessity of adding a learning center instructor, and the requirement to 
add numerous lectures to compensate for inadequate workbooks. The 
currency cost is higher than anticipated as a result of underestimating 
the revision effort. Revisions were required far more often and required 
the efforts ct many more workers than planned for. 

The cost differences between VP-3l's present costs and the “what 
Ssroulad have been costs" based on the original design indicate that the ISD 
system that chose the workbook method for the primary means of 
instruction 1s flawed because the workbooks were not entirely satisfactory 
tor instruction. The ISD program failed to allow for the boredom 
associated with studying workbooks for hours on end without student- 
instructor interaction. Implementation of the workbook system failed to 
take into account the ramifications of applying the rapidly changing 
information to a fixed and difficult to change medium. 

The ISD system could also have problems because it is not well 
adapted to the realities of the military. The ISD system uses a few 
experts to organize and lead a team of subject matter experts to develop 
a training program, but once the program is developed it is used in a 
command composed mostly of transient military people who are given the 


responsibility to maintain the curriculum as it was designed but who are 
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unfamiliar with ISD. Within a few years of introducing the P-3 ISD 
designed training program, few if any of the major changes made had been 
validated. Even more problematic is that during those early years fleet 
reports of poorly trained students motivated many of the changes which 
did occur. This raises the question of whether or not the original ISD 
program was more effective than the program it replaced. The training 
programs now in place at VP-31 and VP-30 are meeting the fleet needs 
through training programs that have evolved considerably from the 
original ISD design, and those changes were made by instructors doing 
what they thought was best without heavy development, implementation, 
and valication costs. 

Because millions of dollars are spent by the Navy on ISD programs, 
whether or not training effectiveness is increased as a result of using the 
ISD process to design a curriculum would be an interesting future 
research topic. Perhaps it is better to save the millions of dollars and 
instead rely on the intuition and expertise of the managers and instructors 
in the training program to make changes as needed, 

When selecting training media alternatives, a cost-benefit analysis that 
compares the opportunity costs associated with the alternatives should be 
used as the measure for comparison. To be accepted, a new method of 
instruction should compare favorably with conventional instruction. A 
cost-benefit analysis that evaluates a new technology should be careful not 
to underestimate the effort required to maintain the currency of the 
material presented by the technology and not underestimate the manpower 


demands of using the new technology. 
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